Effective red eye removal in digital images without face detection

ABSTRACT

Methods and apparatuses for detecting and correcting red-eye in an image are provided. In one aspect, red-eye can be detected without detecting faces. Red-eye candidates are identified and eliminated by evaluating a series of statistics, including color, size, shape, distance, etc. A plurality of regions, including an eye region, a pupil region, an iris region, and a sclera region can be associated with each red-eye candidate. Color distribution statistics are computed with respect to the presence of target color, glint, skin, or other-color pixels in the plurality of regions. Red-eye candidates may be paired with other red-eye candidates for further processing. Further statistics may be computed based on the location of the red-eye candidates in the image, and the location of each red-eye candidate relative to other red-eye candidates. Pixels associated with selected red-eye candidates may be color adjusted.

FIELD

The disclosure relates to eye color correction in digital images. Inparticular, the disclosure relates to techniques for detecting andremoving red-eye based on color distribution statistics of red-eyecandidates, and other parameters such as size, location relative to animage's borders, and location relative to other red eye candidates.

BACKGROUND

The problem of red-eye is well-known in photography. Red-eye is causedby light from a camera's flash being reflected off blood vessels in asubject's retina back to the camera. Occurrence of red-eye may increasewhen the subject's pupils are wide open, as in a darkened room.

Various red-eye reduction algorithms have been developed to identifyinstances of red-eye in an image and correct them. Some algorithmsperform face detection, then locate the eyes within a detected face, andfinally correct for the redness of the eye. Face detection algorithms,however, are generally computation-intensive and put great demands on acomputing system's power source. This makes face detection algorithmsundesirable for use in some portable devices such as camera phones,which have limited computational capabilities compared to desktopcomputers, and must make judicious use of battery power.

Disclosed herein are computationally efficient red-eye correctiontechniques well-suited for devices such as camera phones.

SUMMARY

An aspect of the present disclosure provides a method for adjusting eyecolor in a digital image, the method comprising defining a plurality ofcolor correction candidates, each color correction candidate comprisinga plurality of pixels; classifying each pixel in each color correctioncandidate as either a target color pixel, or a glint pixel, or a skinpixel, or an other-color pixel; computing color distribution statisticsfor each color correction candidate based on the classified pixels;identifying preferred color correction candidates based on the computedstatistics; and adjusting the color of the preferred color correctioncandidates.

Another aspect of the present disclosure provides a method for adjustingeye color in a digital image, the method comprising defining a pluralityof color correction candidates, each candidate comprising a plurality ofpixels; defining regions comprising a pupil region (PR), an eye region(ER), an iris region (IR), and a sclera region (SR) for each colorcorrection candidate; computing likelihood statistics based on theregions for each color correction candidate; comparing a colorcorrection candidate with another color correction candidate based onthe computed likelihood statistics; identifying a preferred colorcorrection candidate based on the comparing; and adjusting the color ofpixels in the preferred color correction candidate.

Yet another aspect of the present disclosure provides a method foradjusting eye color in a digital image, the method comprising defining aplurality of color correction candidates, each candidate comprising aplurality of pixels; defining a pupil region (PR) for each colorcorrection candidate; performing at least one test on each of theplurality of color correction candidates; comparing a color correctioncandidate with another color correction candidate based on the result ofsaid at least one test; identifying a preferred color correctioncandidate based on the comparing; and adjusting the color of pixels inthe preferred color correction candidate.

Yet another aspect of the present disclosure provides a method foradjusting eye color in a digital image, the method comprising defining aplurality of color correction candidates, each color correctioncandidate comprising a plurality of pixels; for each color correctioncandidate, computing a match metric between the color correctioncandidate and every other color correction candidate; based on the matchmetric, assigning a complementary color correction candidate to eachcolor correction candidate, each color correction candidate and assignedcomplementary color correction candidate forming a pair; comparing acharacteristic of a first pair with a second pair to determine apreferred pair; and adjusting the color of pixels in the colorcorrection candidate and the complementary color correction candidate ofthe preferred pair.

Yet another aspect of the present disclosure provides a method foradjusting eye color in a digital image, the method comprising defining aplurality of color correction candidates, each candidate comprising aplurality of pixels; for each color correction candidate, selecting acomplementary color correction candidate, the color correction candidateand the complementary color correction candidate forming a pair;assigning a failure metric to each pair; defining a first region in thedigital image having a first spatial relationship relative to a firstpair; and comparing the failure metric of any pair lying in the firstregion to the failure metric of the first pair.

Yet another aspect of the present disclosure provides a method foradjusting eye color in a digital image, the method comprising defining aplurality of color correction candidates, each candidate comprising aplurality of target color pixels; performing a plurality of tests oneach color correction candidate; for each color correction candidate,accumulating a number of failed match points (FMP's) based on a resultof each test performed; and comparing the accumulated FMP's of at leastone color correction candidate to the accumulated FMP's of at least oneother color correction candidate.

Yet another aspect of the present disclosure provides a method foradjusting eye color in a digital image, the method comprising defining aplurality of color correction candidates, each candidate comprising aplurality of pixels; computing an average size of the color correctioncandidates; identifying a color correction candidate having a sizeoutside a range that is a function of the average size; and notadjusting the color of pixels in said identified color correctioncandidate.

Yet another aspect of the present disclosure provides a method foradjusting eye color in a digital image, the method comprisingidentifying target color pixels in the digital image; defining colorcorrection candidates based on the target color pixels; defining aregion map for each color correction candidate, the region mapcomprising at least one of an eye region, a pupil region, an irisregion, and a sclera region; computing statistics for each colorcorrection candidate based on its region map; identifying preferredcolor correction candidates based on the computed statistics; andadjusting the color of pixels in the preferred color correctioncandidates.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows an embodiment of an imaging system using red-eye correctiontechniques described herein.

FIG. 2 shows an embodiment of the red-eye correction techniques that canbe used in block 106 of FIG. 1.

FIGS. 3-3A show an embodiment of the system depicted in FIG. 2.

FIG. 4 depicts an example of a connected cluster (CC).

FIG. 4A depicts an example of a cluster containing a large number oftouching pixels.

FIG. 5 illustrates an embodiment wherein a first CC (CC1) and a secondCC (CC2) are within a minimum threshold distance from each other.

FIGS. 6-6D illustrate an embodiment of the various regions definedherein.

FIG. 7 depicts how a pupil region (PR) and iris region (IR) can bedefined from an eye region (ER) according to an embodiment.

FIG. 8 depicts the various regions with reference to underlying pixels.

FIG. 9 shows a mouth region of an REC-CREC (Red-EyeCandidate—Complementary Red-Eye Candidate) pair.

DETAILED DESCRIPTION

Disclosed herein are techniques for correcting red-eye in an image,without necessarily resorting to face detection algorithms. Note thataspects of the present disclosure may nevertheless be used inconjunction with face detection algorithms, or any other red-eyecorrection algorithms, to enhance their performance.

FIG. 1 shows an embodiment of an imaging system using red-eye correctiontechniques described herein. FIG. 1 shows a camera 110, which may beembodied in a digital camera, a camera phone, or any other image capturedevice. Camera 110 may be provided with a flash 112 for illuminating asubject 100. Camera 110 may include an image capture component 102 whichgenerates a captured image 104 of the subject 100. In an embodiment, theimage capture component 102 may include a lens and a CMOS, CCD, or someother form of digital pixel array. The captured image 104 may be adigital image defined by a plurality of pixels. The captured image 104can be processed by a red-eye correction block 106 as described in thepresent disclosure, generating the corrected image 108.

Note the embodiment shown in FIG. 1 should not be interpreted to limitthe scope of the present disclosure, as the disclosed techniques maygenerally be applied to correct for red-eye in any digital image. Forexample, the disclosed techniques need not be implemented in a camera110 as depicted in FIG. 1. Rather, some or all of the techniquesdisclosed may be implemented offline, for example, as part of apost-processing software program running on a personal computer (PC).

FIG. 2 shows an embodiment of the red-eye correction techniques that canbe used in block 106 of FIG. 1. Block 150 accepts a digital image, andidentifies regions of the image that are possible red-eye candidates(REC's). In identifying a candidate list, block 150 may add or removecandidates from the list based on certain characteristics such acandidate's location, size, shape, etc., as will be described furtherherein. Block 152 focuses on color distribution characteristics of theidentified REC's to provide more information on the likelihood of eachREC being a legitimate instance of red-eye. Block 154 attempts toidentify a Complementary REC (CREC) for each REC, based on theassumption that red-eye in an image is likely to occur in pairs.Finally, block 156 corrects the color in the REC's and/or CREC'sidentified in the previous steps.

Note other embodiments may differ from that shown in FIG. 2 whileemploying the techniques of the present disclosure. For example, anembodiment may sequence the blocks differently than shown in FIG. 2.Alternatively, an embodiment may add additional processing blocks to, orremove processing blocks from, the ones shown in FIG. 2. Furthermore,alternative blocks may combine aspects of separate blocks in a singleblock, e.g., color distribution statistic collection and REC refinementin a single block.

FIG. 3 shows one embodiment of the system depicted in FIG. 2. Thetechniques depicted by blocks 202-228 may collectively function toidentify the presence of red-eye in the captured image 104 of FIG. 1,and remove the identified red-eye to produce corrected image 108. Thedetails of each block will be described herein. Note that the techniquesas disclosed need not be limited to embodiments comprising all of theblocks shown in FIG. 3. Embodiments adopting various subsets of theblocks shown are contemplated as well. Also contemplated are embodimentswherein the order of the blocks is different from that depicted in FIG.3. Unless otherwise noted, the processing techniques disclosed maygenerally be sequenced differently than depicted in any disclosedexemplary embodiments.

In FIG. 3, block 200 downsamples an input digital image to reduce thecomplexity of the subsequent computations. Downsampling is particularlysuitable for embodiments of the disclosure in portable devices such ascamera phones, where power can be a limited resource. In an embodiment,the image can be downsampled to ¼ of its original size by uniformlysampling every one out of four pixels of the original image. Note inother embodiments, block 200 need not be implemented, and the steps canproceed by treating the entire image rather than a downsampled versionof the image.

Blocks 202-214 in FIG. 3 depict one embodiment of the block 150 in FIG.2 for identifying and refining Red Eye Candidates (REC's) in an image.

Block 202 depicts a skin tone and red tone detection block. Skin tonedetection identifies areas of the captured image 104 that have a colorassociated with human skin tones. The detection of image portions likelyto be human skin has been treated in the prior art. Representativetechniques are disclosed in U.S. Pat. App. Pub. No. 2007-0104472,entitled “Skin Color Prioritized Automatic Focus Control ViaSensor-Dependent Skin Color Correction,” assigned to the assignee of thepresent invention and incorporated herein by reference. In anembodiment, any technique for skin tone detection, including thosetaught in the prior art, may be used to perform the functions of block202. In an embodiment, skin tone pixels (or “skin pixels”) mayspecifically be identified as any pixels in an image having values inthe hue saturation value (HSV) color space noted in Table 1. Note allvalues given in Table 1 are on an 8-bit scale, i.e., 0 to 255.

TABLE 1 Skin Hue Skin Saturation Skin Value Minimum 1 7 40 150 Maximum 119 175 245 Minimum 2 7 60 120 Maximum 2 19 160 150

Table 1 effectively specifies three HSV color ranges for skin pixels.For example, a pixel having HSV value (10, 100, 130) may be deemed to bea skin pixel because it falls within the HSV range defined by Minimum 2and Maximum 2 in Table 1.

Note other embodiments may employ different ranges for skin tone thandepicted in Table 1, or use color representations other than HSV, suchas red-green-blue (RGB) or luma-chroma (YCbCr).

In contrast with skin tone detection, the red tone detection referencedin block 202 identifies pixels having reddish color commonly found ininstances of red-eye. In an embodiment, the red tone may be identifiedas any pixel having HSV ranges as noted in Table 2. Again, all valuesgiven in Table 2 are on an 8-bit scale, i.e., 0 to 255.

TABLE 2 Red Hue Red Saturation Red Value Minimum 1 230 130 93 Maximum 1255 227 255 Minimum 2 0 130 93 Maximum 2 9 227 255 Minimum 3 220 110 115Maximum 3 255 150 130 Minimum 4 0 110 115 Maximum 4 5 150 130

Note other embodiments may employ different ranges for red tone thandepicted in Table 2, or use different representations of color otherthan HSV, such as red-green-blue (RGB) or luma-chroma (YCbCr). Note alsothat aspects of the present disclosure can be applied to correct notonly for red-eye but for other color aberrations found in photographicimages, such as blue-eye, yellow-eye, or green-eye. This can easily bedone by replacing the ranges specified in Table 2 with alternativevalues.

Block 204 depicts a block for creating connected clusters (CC's) andconnected cluster rectangles (CCR's). This block identifies connectedclusters (CC's) made up of touching red pixels (or other target colorpixels) identified from block 202. The block then identifies, labels,and computes a connected cluster rectangle (CCR) for each CC identified.FIG. 4 depicts an example of a connected cluster (CC). Each of thesquares in FIG. 4 represents a pixel, and each shaded square representsa red pixel. The cluster of pixels labeled “CC” has been identified ascomprising a cluster of touching red pixels.

For each identified CC, a corresponding connected cluster rectangle(CCR) can be defined as the minimum-sized rectangle which entirelyencloses the identified cluster. The connected cluster rectanglecorresponding to the labeled CC is labeled “CCR” in FIG. 4. Notealternative embodiments may define shapes other than a rectangle torepresent the identified CC's for simplified processing, such as acircle or trapezoid or other parallelograms.

In an embodiment, clusters of touching pixels can be identified using arecursive “flood fill” algorithm well-known in the prior art todetermine all nodes connected to an arbitrary node in a pixel array.Other embodiments may employ other techniques to identify clusters oftouching pixels.

In an embodiment, to prevent overrunning of the memory stack inimplementing the flood fill algorithm, or any other recursive algorithm,a check can be performed on the number of recursions performed. If thenumber of recursive calls exceeds a threshold, the algorithm may declarea CC based on the pixels already identified as forming a cluster, andsubsequently the algorithm may be started on a new cluster. In anembodiment, the threshold can be set at 3000 calls deep. For example, ifa set of touching red pixels contains 5000 pixels, block 204 mayidentify a CC based on 3000 of those pixels, then identify a separate CCbased on the remaining 2000 pixels. FIG. 4A depicts an example of acluster containing a large number of touching pixels. In FIG. 4A, eachsquare represents a suitably large number of pixels. Upon reaching asize limit in filling the first cluster CC1, the flood fill algorithmcan proceed to create a second cluster CC2. After the CC's areidentified, they can then be associated with corresponding CCR's asdescribed earlier.

Block 206 identifies a CCR that is within some threshold distance toanother CCR, and combines those CCR's to form a new composite CCR. In anembodiment, the distance between CCR's is defined as the distancebetween the center of one CCR and the center of another CCR. In anembodiment, 12 pixels can be used as the threshold distance. In otherembodiments, any other measure and/or threshold for distance may bedefined. For two CCR's that are determined to be close to each other, anew CCR can be defined that is the minimum-sized rectangle that enclosesboth of the previous CCR's, and the previous CCR's are eliminated. FIG.5 illustrates an embodiment wherein a first CC (CC1) and a second CC(CC2) are within a minimum threshold distance from each other. Here, thedistance between the CC's is measured as the minimum distance betweenthe centers of each CC. CCR1 and CCR2 are eliminated and replaced by aCCRNEW that encloses both CCR1 and CC2.

Block 208 identifies any CCR that has at least one pixel overlappingwith another CCR. In an embodiment, for any CCR that overlaps anotherCCR, the smaller of the two CCR's is eliminated. In other embodiments, anew CCR can be defined that encloses a set of CCR's that shareoverlapping pixels.

Block 210 removes CCR's that have shapes and/or sizes that make themunlikely to be images of pupils. In an embodiment, block 210 removesCCR's having a total size (e.g., total number of pixels) less than 3pixels or greater than ((image_width/256) * 6)², where image-width is awidth of the total image in pixels. Also, block 210 can remove any CCRlarger than 2000 pixels, wherein the CCR also has an aspect ratio (i.e.,length divided by height) less than 0.5 or greater than 2.4. In otherembodiments, other limits for sizes may be defined. In otherembodiments, alternative metrics may be used to identify any otherunlikely shapes, sizes, and/or dimensions.

Block 212 expands each CCR into a region known as a Red Eye Candidate(REC). In an embodiment, the expansion of a CCR into an REC is done toextract additional information from the region surrounding theidentified red pupils, including areas associated with the iris, thesclera, and the surrounding skin. In an embodiment, the expansion can bedone according to a look-up table as follows:

TABLE 3 Maximum of CCR width and height (in pixels) 1 2 3 4 5 6 7 to 1516 to 28 29 and above Expansion 16 16 13 10 8 7 5 4 3 multiplier

In the look-up table above, the maximum of a CCR's width and height inpixels determines a multiplier that is multiplied to the dimensions ofthe CCR to expand the size of the CCR into an REC. For example, a CCRhaving a width of 2 and a height of 4 will be expanded into an REChaving a width of 20 and a height of 40 (i.e., both dimensionsmultiplied by 10) according to the look-up table. In an embodiment, thepost-expanded REC may include the pre-expanded CCR at its center, asdepicted in FIG. 5A. Note the expansion shown in FIG. 5A is not doneaccording to the expansion multipliers given in the table above, and isgiven only to illustrate the expansion of a CCR to an REC. In general,alternative multipliers may be substituted for those given in thelook-up table above. Note alternative embodiments may employ functionalmappings between CCR and REC other than those given in the table above.Such functions may be implemented using look-up tables or any othercomputational methods.

Block 214 searches an area suitably larger than an REC to detect ifthere is skin surrounding the REC. If there is not enough skin detectedaround the REC, the REC may be eliminated. This operation is based on anassumption that legitimate instances of red-eye will likely besurrounded by skin. In an embodiment, for each REC, the block can searchan enlarged area four times larger than that REC (centered on thatoriginal REC), and remove any REC whose enlarged area contains aproportion of skin pixels less than 4/9. Other embodiments may definealternative search areas and skin detection thresholds. In anembodiment, skin pixels can correspond to those identified from theskin-tone detection block 202.

After expansion of CCR's to REC's, a series of statistics can becollected from the remaining REC's that provide information on thelikelihood that a particular REC is a legitimate instance of red-eye. Inan embodiment, a set of statistics may be defined with reference to aneye region (ER), pupil region (PR), iris region (IR) and sclera region(SR).

In this specification and in the claims, the ER is defined to be theentire set of pixels associated with an REC. The PR is defined to be aninnermost subset of the pixels in an REC. The IR is defined to be aregion that completely surrounds the PR. The SR is defined to be aregion that completely surrounds the IR. In an embodiment, the union ofthe PR, IR, and SR form the ER, i.e., the PR, IR, and SR arenon-overlapping, and they collectively account for all the pixels withinthe ER. In another embodiment, the PR may overlap slightly with the IR,and the IR may overlap slightly with the SR. In yet another embodiment,there may be a gap between the border of the PR and the inner border ofthe IR, and/or a gap between the outer border of the IR and the innerborder of the SR, and/or a gap between the outer border of the SR andthe border of the ER. Note the ER, PR, IR, and SR may generally have anyshape, regular or irregular, and need not be limited to having theshapes specifically disclosed herein.

FIG. 6 illustrates an embodiment of the various regions defined. NoteFIGS. 6-6D and the accompanying descriptions are provided forillustrative purposes only, and are not meant to limit the definitionsof the four regions as given above. As depicted in FIG. 6A, an eyeregion (ER) may generally cover one eye region of the human face,preferably including most of the bottom and top eyelid. As depicted inFIG. 6B, the pupil region (PR) may generally cover the pupil of an eye,and may contain pixels having colors associated with pupils such asblue, brown or green, as well as the target color to be corrected. Asdepicted in FIG. 6C, the iris region (IR) may generally contain the irisof the eye, and may cover the portion of the eye where light from thecamera flash is absorbed. The IR may contain the color black, as well asred due to reflection from the cornea. As depicted in FIG. 6D, a scleraregion (SR) may generally cover only the white portion of the eye. TheSR may also cover some part of the upper or lower eyelids.

In a specific embodiment, the definitions of the PR and IR may befurther refined, with reference to FIG. 7. First, for each row of theER, count the number of red pixels present in each row, as shown inblock 700. This constitutes a one-dimensional projection of the redpixels in the ER along each row of the y- (or vertical) axis. Next, foreach column of the ER, count the number of red pixels present in eachcolumn, as shown in block 702. This is a one-dimensional projection ofthe red pixels in the ER along the columns of the x- (or horizontal)axis. For the y-axis projection, find the row having the maximum numberof red pixels, and denote the number of pixels in that row as max_(row),as shown in block 704. For the x-axis projection, find the column havingthe maximum number of red pixels, and denote the number of pixels inthat column as max_(cols). Divide max_(rows) by 3, and max_(cols) by 3,to obtain max_(rows)/3 and max_(cols)/3, respectively. Note alternativeembodiments may utilize any divisor other than 3.

In an embodiment, the borders of the PR may then be defined as follows.The PR may be bounded on the top by the first row (counting from the toprow of the ER) having a number of red pixels equal to or greater thanthe number max_(rows)/3 previously computed. Similarly, the PR may bebounded on the bottom by the last row (counting from the top row of theER) having a number of red pixels equal to or greater than the numbermax_(rows)/3 previously computed. The PR may be bounded on the left bythe first column (counting from the leftmost column of the ER) having anumber of red pixels equal to or greater than the number max_(cols)/3.Similarly, the PR may be bounded on the right by the last column(counting from the leftmost column of the ER) having a number of redpixels equal to or greater than the number max_(cols)/3.

FIG. 8 depicts the various regions with reference to underlying pixels.In an embodiment, the borders of the IR may be determined as follows.According to FIG. 8, which shows a PR already defined, the IR's innerborder may be taken to be the set of pixels surrounding the outer borderof the PR. The IR's outer border may be defined as the set of pixelslocated halfway (or approximately halfway) between the borders of the PRand the ER. Note alternative embodiments may define the IR differently,for example, as being closer to or farther away from the PR and ERborders than described herein.

In an embodiment, the SR may be defined as the region between the outerborder of the IR and the border of the ER, as shown in FIG. 8.

In another specific embodiment, the PR, IR, and SR can be defined basedon “weighted” versions of the number of red pixels counted in each rowand column. This may, for example, serve to emphasize finding convexhulls of reds about a center of the ER, as opposed to the entire ER. Inan embodiment, the count of red pixels for each row and column may beinitialized with negative numbers that decrease in value (i.e., becomemore negative) away from the center. This initialization effectivelyassigns greater weight to red pixels near the center of the ER than tored pixels at the periphery of the ER, which may be present due tonon-red-eye related effects such as, for example, excessive sun exposureor alcohol consumption. In an embodiment, the counts may be initializedwith any numbers (negative and/or non-negative) that decrease in valueaway from the center.

Based on a defined ER, PR, IR, and SR for each REC, a series ofstatistics can be defined to gather further information on whether theREC is a legitimate instance of red-eye. Note various embodiments mayutilize all or only a subset of the statistics disclosed. Theinformation derived from each statistic may be quantified, weighted,and/or summed to obtain an aggregate statistic or statistics used forfurther processing. Note alternative embodiments may define the ER, PR,IR and SR in ways other than according to the specific embodiment asgiven in the refined definition above. Such alternative embodiments mayapply the statistics herein described in altered form, for example byvarying the thresholds and/or metrics associated with the statistics.

In an embodiment, the statistics can be characterized as a series oftests based on the likely properties of an REC containing red-eye. In anembodiment, for every REC that fails one of these tests, a number offailed match points (FMP's) may be accumulated for that particular REC.The net accumulated FMP's for each REC may be used to indicate howlikely the REC is to contain red-eye. In an embodiment, the greater thenumber of FMP's for an REC, the less likely the REC is to containred-eye. Note alternative embodiments may assign different weights tothe result of each test than those given herein, e.g., by accumulatingdifferent numbers of FMP's for failing each test.

In an embodiment, the statistics collected can relate to thedistribution of colors within a region that contains the REC. In anembodiment, the presence of four types of colors may be determined ineach REC: skin, red, glint, and other-color (or “OC,” corresponding topixels that are not skin, red, or glint). In an embodiment, the numbersof red, skin, glint and OC pixels in the PR, IR, SR, and ER of each RECare counted. In an embodiment, the ranges of colors for skin and red maybe as defined earlier in Tables 1 and 2, respectively. In an embodiment,the range of colors for glint may be any HSV pixel having S equal to orless than 100 (on an 8-bit scale), as well as having V equal to orgreater than 190. The collection of color distribution statistics isdepicted in block 216 of FIG. 3A, which is a continuation of FIG. 3.

Described herein is an embodiment wherein a series of tests may be usedto gather information on the presence of red-eye in an REC. If an RECfails a particular test, a number of failed match points (FMP's) may beaccumulated to a running total associated with that REC. Note variousembodiments may employ some or all of these tests, and may also employother statistics not explicitly described herein. Various embodimentsmay perform the tests using other thresholds, weights, multipliers,factors, and weights (e.g., number of FMP's accumulated) than thosedisclosed for the exemplary embodiments. Note that the tests are notmeant to be interpreted as absolute statements about the characteristicsof legitimate instances of red-eye. Rather, they are intended to providea number of metrics to help assess whether an REC is more or less likelyto be a legitimate instance of red-eye. Note the descriptors “first,second, third, etc.,” for the tests are used solely as identifiers, andare not meant to imply that any test must be followed by any other testin a preferred sequence, or that an embodiment must apply any particularnumber of these tests.

According to a first test, a pupil that is too small in size is unlikelyto be an instance of red-eye needing correction. In an embodiment, ifthe number of pixels in a PR is less than 16, accumulate 3 FMP's.

According to a second test, the border of an IR should be separated fromthe border of an SR by some minimum distance. In an embodiment, if aborder of an IR is less than n pixels from a border of a correspondingSR, where n is defined as max (3, width of SR/7), then instantlydisqualify the REC, e.g., accumulate a maximum number of FMP's thatautomatically fails the REC. Other embodiments may define n in otherways.

According to a third test, the PR likely has an aspect ratio, i.e.,length divided by height, that falls within a predetermined range. In anembodiment, if the PR's aspect ratio is not within certain look-up-table(LUT)-based tolerances, then accumulate 2 FMP's. In an embodiment, theLUT is as given below.

TABLE 4 Size Of PR (# of pixels) Min Aspect Ratio Max Aspect Ratio  <2990.20 5.0 300-399 0.30 4.0  400-2600 0.40 3.0 2700-2899 0.45 2.5 >29000.45 2.0

According to a fourth test, the ER should contain some minimumproportion of skin pixels. In an embodiment, if the number of skinpixels in an ER is less than 27% of all pixels in the ER, thenaccumulate 3 FMP's.

According to a fifth test, the skin pixels in the ER should be foundmostly in the SR. In an embodiment, the number of skin pixels in the SRis divided by the number of skin pixels in the ER. If the quotient isless than 0.44, accumulate 3 FMP's.

According to a sixth test, the number of skin pixels in the PR should beproportionately low as compared to the ER. In an embodiment, the numberof skin pixels in the PR is divided by the number of skin pixels in theER. If the quotient is greater than 0.01, accumulate 3 FMP's.

According to a seventh test, if the number of skin pixels in the PR isgreater than half the number of red, glint, and OC pixels in the PR,then accumulate 3 FMP's.

According to an eighth test, the OC pixels in the SR should number nomore than a certain factor times the number of OC pixels in the PR andIR combined. In an embodiment, if the number of OC pixels in the SR isgreater than four times the number of OC pixels in the PR and IR, thenaccumulate 3 FMP's.

According to a ninth test, there should be comparatively more red andglint than OC pixels in the PR. In an embodiment, if the number of OCpixels in the PR is greater than 3/2 times the number of red and glintpixels in the PR, accumulate 1 FMP.

According to a tenth test, there should be some significant number ofred pixels in the PR in a legitimate instance of red-eye. In anembodiment, if the number of pixels in the ER is greater than 400, thenthere should be at least 20 red pixels in the PR. Otherwise, accumulate2 FMP's.

According to an eleventh test, there should be some significant numberof red pixels in the PR. In an embodiment, this test is only applied ifthe PR size is larger than a threshold. In an embodiment, if the numberof pixels in the PR is greater than 200, and the number of red pixels inthe PR divided by the number of pixels in the PR is less than 40%, thenaccumulate 1 FMP.

According to a twelfth test, the red and glint pixels in the PR shouldaccount for most of the pixels in the PR. In an embodiment, this test isonly applied if the PR size is larger than a threshold. In anembodiment, if the number of pixels in the PR is greater than 200, andthe number of red, glint pixels in the PR divided by the number ofpixels in the PR is less than 80%, then accumulate 1 FMP.

According to a thirteenth test, the red pixels in the PR should accountfor some sizable portion of the total red pixels in the ER. In anembodiment, if the number of red pixels in the PR is less than thenumber of red pixels in the ER divided by 3, accumulate 1 FMP.

According to a fourteenth test, there should be fewer red pixels in theIR than in the PR. In an embodiment, if the number of red pixels in IRis greater than the number of red pixels in PR, accumulate 1 FMP.

According to a fifteenth test, there should be more red pixels in the PRthan in the IR and SR combined. In an embodiment, if the number of redpixels in the PR is less than one-third of the number of red pixels inthe IR and SR combined, then accumulate 3 FMP's.

According to a sixteenth test, the glint and OC pixels in the PR and IRshould account for some minimum number of the total pixels in the PR andIR. In an embodiment, if the number of glint, OC pixels in the PR and IRdivided by the number of pixels in the PR and IR combined is less than3%, accumulate 1 FMP.

According to a seventeenth test, there should be some minimum number ofglint pixels present in the PR and IR. Glint is caused by the reflectionof the camera's flash from a pupil. In an embodiment, if the number ofglint pixels in PR and IR is less than 3, accumulate 2 FMP's.

According to an eighteenth test, the number of glint pixels should notexceed some maximum. In an embodiment, if the number of glint pixels inthe ER is greater than eight times the number of red pixels in the ER,accumulate 2 FMP's.

According to a nineteenth test, for some suitably large ER, there shouldbe at least a certain number of glint pixels in the PR and IR. In anembodiment, if the number of pixels in the ER is greater than 10,000,and the number of glint pixels in the PR and IR is less than 25,accumulate 2 FMP's.

When FMP's for a predetermined set of tests are accumulated for eachREC, the cumulative FMP's for each REC can be compared against athreshold to determine whether the REC will be eliminated as acandidate. In an embodiment wherein all the tests disclosed herein areused to accumulate FMP's, a threshold of 2 can be used. In alternativeembodiments, where information collected from color distributionstatistics are quantified, weighted, and/or accumulated according to adifferent scheme, a different threshold can be chosen to eliminate REC'sfrom consideration.

In an embodiment, to increase the accuracy of the color distributionstatistics, the operations starting from block 216 in FIG. 3A can beperformed on the full, non-subsampled, digital image, as opposed to thesubsampled version. This is represented as block 215 in FIG. 3A. Notethe boundaries and locations of the REC's (possibly determined based onthe subsampled image) may need to be redrawn and resealed accordingly tocorrespond to the non-subsampled image. Note that other embodiments maycontinue to perform operations on the subsampled image following block216, or on an image subsampled by a different factor. In general,switching between a subsampled and non-subsampled image for processingis a trade-off between accuracy and computational complexity. In someembodiments, the switching can be performed at any stage of theembodiment depicted in FIGS. 3, 3A, and also at any stage of otherembodiments not depicted in FIGS. 2 or 3, 3A. Switching can also be donefrom a non-subsampled image to a subsampled image for particularlycomputationally intensive stages.

Referring back to FIG. 3A, block 218 identifies REC's that overlap oneor more other REC's remaining after eliminating color distributionstatistics. Block 218 eliminates the overlapping REC with the higher orhighest accumulated value of FMP's. In an embodiment, if two (or more)such REC's have the same value of FMP's, then the one positioned lower(or lowest) in the image can be eliminated.

Block 220 identifies a Complementary REC (CREC) for each REC. This isbased on the assumption that instances of red-eye likely appear inpairs. In an embodiment, a CREC is chosen for each REC. To choose aCREC, a bad match metric (BMM) can be computed between each REC andevery other REC. In an embodiment, a BMM can be computed as a weightedsum of several factors. One factor may be the slope between an REC and acandidate CREC. In an embodiment, the slope between two REC's can becomputed as the slope of a line connecting corresponding points on thetwo REC's, e.g., the top-leftmost corners. Another factor may be thedistance between the centers of the REC and the candidate CREC. Anotherfactor may be the FMP value of a candidate CREC. Another factor may bethe size differential between the REC and a candidate CREC. Anotherfactor may be whether the candidate CREC has had the opportunity to findits own CREC, and if so, whether the candidate CREC chose the REC underexamination as its CREC. In an embodiment, weights are assigned to eachof these factors to obtain a BMM for each candidate CREC. The candidateCREC with the lowest BMM is selected as the CREC for the REC. Note inother embodiments, fewer or more criteria may be used to compute the BMMfor a candidate CREC.

In an embodiment, the above factors may be combined as follows to yielda single BMM_(C1-C2), the BMM between REC (C1) and a candidate CREC(C2):

BMM _(C1-C2)=[1+1000*(slope_(C1-C2))]*dist_(C1-C2)*sizeDiff_(C1-C2)*FMPdiff_(C1-C2);

where slope_(C1-C2) is the slope between C1 and C2;

dist_(C1-C2) is the absolute distance between C1 and C2;

sizeDiff_(C1-C2) is the absolute difference in size between C1 and C2;and

FMPdiff_(C1-C2) is the absolute difference in FMP values between C1 andC2.

In an embodiment, the computed BMM_(C1-C2) is then further discounted by10% if C2's computed CREC happens to be C1. In an embodiment,sizeDiff_(C1-C2) can be computed by dividing the size of the larger ofC1 and C2 by the size of the smaller of C1 and C2. Note the BMMcomputation above is given for illustrative purposes only. Otherembodiments may alternatively combine the factors disclosed to computeother BMM's.

In an embodiment, once a CREC has been identified for every REC, thenthe following assumptions may be tested. First, it is assumed that anREC should be paired up with a CREC that considers that REC to be itsCREC, i.e., a reflexivity assumption. In an embodiment, if an REC'sCREC's CREC is not the original REC, then eliminate that REC if its FMPis above some threshold. In an embodiment, the threshold is 2. Second,if the reflexivity assumption holds for an REC-CREC pair, then testfurther assumptions to make sure the pairing is feasible. According toone assumption, an REC should not be located too far away from its CRECin an image. In an embodiment, the REC and its CREC can be tested tomake sure they are not more than five times further apart than theshorter of their two widths. According to another assumption, the sizedifferential between an REC and its CREC should not be great. In anembodiment, the REC's in an REC-CREC pair can be tested to make sureneither has a size three times greater than the other. Third, if an RECis left without a CREC after testing the above assumptions, theneliminate the REC if its FMP is above some threshold. In an embodiment,the threshold may be 2. Note in an embodiment, the assumptions disclosedin this paragraph need not be tested separately from the computation ofthe BMM. The assumptions can also be incorporated in the computation ofthe BMM itself.

Block 222 eliminates REC-CREC pairs based on their locations relative toother pairs. In an embodiment, for every REC-CREC pair, determine aminimum FMP of the pair. For each pair, the minimum FMP of the pair maybe defined as min (FMP of REC, FMP of CREC), and can be referred to astopFMP. Then, identify a “mouth” region spatially beneath the REC-CRECpair. In an embodiment, the mouth region of an REC-CREC pair can bedefined as the entire set of pixels in the image spatially beneath arectangular box that entirely encloses the pair. In an embodiment, therectangular box is defined as the minimum-sized rectangle entirelyenclosing the REC-CREC pair, plus two width adjustments. In the firstwidth adjustment, the left side of the minimum-sized rectangle isfurther extended to the left by an amount equal to the width of theleftmost candidate of the REC-CREC pair. In the second width adjustment,the right side of the minimum-sized rectangle is further extended to theright by an amount equal to the width of the rightmost candidate of theREC-CREC pair. FIG. 9 depicts a mouth region defined according to suchan embodiment. Alternative embodiments may define the mouth region inother ways.

In an embodiment, for any REC that lies fully within the mouth region,eliminate the REC lying in the region and its respective CREC if theminimum FMP of the REC-CREC lying in the mouth region, known as botFMP,is more than or equal to topFMP. If, however, botFMP is less thantopFMP, eliminate the REC-CREC that was used to create the mouth region,i.e., the pair associated with topFMP.

Block 224 further eliminates REC's if their size is significantlydifferent from an average REC size, computed over all non-eliminatedREC's. This is based on the assumption that the size of all legitimateREC's in a photo should be similar. In an embodiment, REC size iscomputed as the width times height. In an embodiment, REC's areeliminated if their size is greater than three times the average RECsize, or less than one-third the average REC size. In an embodiment, theeliminated REC's CREC is also eliminated. Note this block can be runmultiple times until there are no more REC's to be eliminated.

Block 226 eliminates REC's that are not within some minimum verticaldistance of the REC in an image having minimum FMP. This is based on theassumption that any occurrence of red-eye in an image will generally bevertically close to other occurrences of red-eye. In an embodiment,REC's are eliminated that are not within +/− imageHeight/4 of thevertical position in the image of the REC with the lowest FMP score,where imageHeight is the height of the image in pixels. In anembodiment, if there is more than one REC having a lowest FMP, the RECpositioned higher in the image may be chosen to break the tie.

Block 228 assumes that the remaining REC's contain legitimate red-eye,and proceeds to correct the pixels in the REC's by adjusting theircolor. Note once a legitimate REC has been identified, any suitablealgorithm may be applied to correct its color. In an embodiment, thecolors are adjusted as follows. All pixels in a remaining REC's PR andIR are color adjusted, except for glint pixels that commonly correspondto the reflection of camera flash. In an embodiment, HSV pixels may beadjusted as follows:

Saturation_(new) =P _(red)*Saturation_(old)+(1−P_(red))*Saturation_(old)/2;

where Saturation_(old) and Saturation_(new) are the saturation of thepixel before and after adjustment, respectively; and P_(red) is thepercentage of red pixels identified in a neighborhood of the pixel ofinterest. In an embodiment, the neighborhood may be a 9 pixel by 9 pixelsquare centered on the pixel of interest. For example, if the pixelunder investigation has HSV color values of (9, 150, 200), and there are34 out of 81 red pixels in a 9×9 square search space around the pixel,then the new value for the pixel would be 107, computed as 42% (or34/81) times the original saturation of 150, added to 58% (or 47/81)times 75, which is the original saturation cut in half. The lowersaturation value tends to make the reddish pixel less red.

The previous description of the disclosed embodiments is provided toenable any person skilled in the art to make or use the presentinvention. Various modifications to these embodiments will be readilyapparent to those skilled in the art, and the generic principles definedherein may be applied to other embodiments without departing from thespirit or scope of the invention. Thus, the present invention is notintended to be limited to the embodiments shown herein but is to beaccorded the widest scope consistent with the principles and novelfeatures disclosed herein.

In one or more exemplary embodiments, the functions described may beimplemented in hardware, software, firmware, or any combination thereof.If implemented in software, the functions may be stored on ortransmitted over as one or more instructions or code on acomputer-readable medium. Computer-readable media includes both computerstorage media and communication media including any medium thatfacilitates transfer of a computer program from one place to another. Astorage media may be any available media that can be accessed by acomputer. By way of example, and not limitation, such computer-readablemedia can comprise RAM, ROM, EEPROM, CD-ROM or other optical diskstorage, magnetic disk storage or other magnetic storage devices, or anyother medium that can be used to carry or store desired program code inthe form of instructions or data structures and that can be accessed bya computer. Also, any connection is properly termed a computer-readablemedium. For example, if the software is transmitted from a website,server, or other remote source using a coaxial cable, fiber optic cable,twisted pair, digital subscriber line (DSL), or wireless technologiessuch as infrared, radio, and microwave, then the coaxial cable, fiberoptic cable, twisted pair, DSL, or wireless technologies such asinfrared, radio, and microwave are included in the definition of medium.Disk and disc, as used herein, includes compact disc (CD), laser disc,optical disc, digital versatile disc (DVD), floppy disk and blu-ray discwhere disks usually reproduce data magnetically, while discs reproducedata optically with lasers. Combinations of the above should also beincluded within the scope of computer-readable media.

1. A method for adjusting eye color in a digital image, the methodcomprising: defining a plurality of color correction candidates, eachcolor correction candidate comprising a plurality of pixels; classifyingeach pixel in each color correction candidate as either a target colorpixel, or a glint pixel, or a skin pixel, or an other-color pixel;computing color distribution statistics for each color correctioncandidate based on the classified pixels; identifying preferred colorcorrection candidates based on the computed statistics; and adjustingthe color of the preferred color correction candidates.
 2. The method ofclaim 1, each target color pixel being a red pixel.
 3. An apparatus foradjusting eye color in a digital image, the apparatus comprising: meansfor defining a plurality of color correction candidates, each colorcorrection candidate comprising a plurality of pixels; means forclassifying each pixel in each color correction candidate as either atarget color pixel, or a glint pixel, or a skin pixel, or an other-colorpixel; means for computing color distribution statistics for each colorcorrection candidate based on the classified pixels; means foridentifying preferred color correction candidates based on the computedstatistics; and means for adjusting the color of the preferred colorcorrection candidates.
 4. A computer program product for correcting eyecolor in a digital image, the product comprising: computer-readablemedium comprising: code for causing a computer to define a plurality ofcolor correction candidates, each color correction candidate comprisinga plurality of pixels; code for causing a computer to classify eachpixel in each color correction candidate as either a target color pixel,or a glint pixel, or a skin pixel, or an other-color pixel; code forcausing a computer to compute color distribution statistics for eachcolor correction candidate based on the classified pixels; code forcausing a computer to identify preferred color correction candidatesbased on the computed statistics; and code for causing a computer toadjust the color of the preferred color correction candidates.
 5. Amethod for adjusting eye color in a digital image, the methodcomprising: defining a plurality of color correction candidates, eachcandidate comprising a plurality of pixels; defining regions comprisinga pupil region (PR), an eye region (ER), an iris region (IR), and asclera region (SR) for each color correction candidate; computinglikelihood statistics based on the regions for each color correctioncandidate; comparing a color correction candidate with another colorcorrection candidate based on the computed likelihood statistics;identifying a preferred color correction candidate based on thecomparing; and adjusting the color of pixels in the preferred colorcorrection candidate.
 6. An apparatus for adjusting eye color in adigital image, the apparatus comprising: means for defining a pluralityof color correction candidates, each candidate comprising a plurality ofpixels; means for defining regions comprising a pupil region (PR), aneye region (ER), an iris region (IR), and a sclera region (SR) for eachcolor correction candidate; means for computing likelihood statisticsbased on the regions for each color correction candidate; means forcomparing a color correction candidate with another color correctioncandidate based on the computed likelihood statistics; means foridentifying a preferred color correction candidate based on thecomparing; and means for adjusting the color of pixels in the preferredcolor correction candidate.
 7. A computer program product for correctingeye color in a digital image, the product comprising: computer-readablemedium comprising: code for causing a computer to define a plurality ofcolor correction candidates, each candidate comprising a plurality ofpixels; code for causing a computer to define regions comprising a pupilregion (PR), an eye region (ER), an iris region (IR), and a scleraregion (SR) for each color correction candidate; code for causing acomputer to compute likelihood statistics based on the regions for eachcolor correction candidate; code for causing a computer to compare acolor correction candidate with another color correction candidate basedon the computed likelihood statistics; and code for causing a computerto identify a preferred color correction candidate based on thecomparing; and code for causing a computer to adjust the color of pixelsin the preferred color correction candidate.
 8. A method for adjustingeye color in a digital image, the method comprising: defining aplurality of color correction candidates, each candidate comprising aplurality of pixels; defining a pupil region (PR) for each colorcorrection candidate; performing at least one test on each of theplurality of color correction candidates; comparing a color correctioncandidate with another color correction candidate based on the result ofsaid at least one test; identifying a preferred color correctioncandidate based on the comparing; and adjusting the color of pixels inthe preferred color correction candidate.
 9. The method of claim 8,further comprising, for each color correction candidate, accumulating anumber of failed match points (FMP's) based on a result of each testperformed; the comparing a color correction candidate with another colorcorrection candidate comprising comparing the accumulated FMP's of onecandidate with the accumulated FMP's of another candidate.
 10. Themethod of claim 9, the at least one test comprising a first testevaluating whether a PR of a color correction candidate has a size lessthan a threshold, the accumulating FMP's comprising accumulating 3 FMP'sif the result of the first test is true.
 11. The method of claim 9, theat least one test comprising a third test evaluating whether a PR of acolor correction candidate has an aspect ratio falling within apredetermined range, the accumulating FMP's comprising accumulating 3FMP's if the result of the third test is false.
 12. The method of claim8, the defining a PR comprising: determining an upper row boundary ofthe PR as the uppermost row having a number of target color pixelsgreater than a first threshold; determining a lower row boundary of thePR as the lowermost row having a number of target color pixels greaterthan a second threshold; determining a left column boundary of the PR asthe leftmost column having a number of target color pixels greater thana third threshold; and determining a right column boundary of the PR asthe rightmost column having a number of target color pixels greater thana fourth threshold.
 13. The method of claim 12, the first and secondthresholds being a predetermined proportion of the maximum number oftarget colors pixels along any row of the color correction candidate,and the third and fourth thresholds being a predetermined proportion ofthe maximum number of target color pixels along any column of the colorcorrection candidate.
 14. The method of claim 12, wherein the number oftarget color pixels in a row or column is weighted.
 15. The method ofclaim 8, the at least one test comprising a first test evaluatingwhether a PR of a color correction candidate has a size less than athreshold.
 16. The method of claim 8, the at least one test comprising athird test evaluating whether a PR of a color correction candidate hasan aspect ratio falling within a predetermined range.
 17. The method ofclaim 8, the at least one test comprising a seventh test evaluatingwhether the number of skin pixels in the PR is greater than apredetermined proportion of the number of target color, glint andother-color pixels in the PR.
 18. The method of claim 17, the targetcolor being red.
 19. The method of claim 8, the at least one testcomprising a ninth test evaluating whether the number of other-colorpixels in a PR of a color correction candidate is greater than apredetermined factor times the number of target color and glint pixelsin the PR.
 20. The method of claim 8, the at least one test comprising atenth test evaluating whether the number of target color pixels in thePR is more than a first minimum threshold.
 21. The method of claim 8,the at least one test comprising an eleventh test evaluating whether thenumber of target color pixels in a PR of a color correction candidate ismore than a predetermined proportion of the total number of pixels inthe PR.
 22. The method of claim 21, the eleventh test further evaluatingwhether the total number of pixels in the PR exceeds a minimumthreshold.
 23. The method of claim 8, the at least one test comprising atwelfth test evaluating whether the combined number of target color andglint pixels in a PR is greater than a predetermined proportion of thetotal number of pixels in the PR.
 24. The method of claim 8, the atleast one test further defined with respect to an eye region (ER). 25.The method of claim 24, at least one test comprising a tenth testevaluating whether the number of target color pixels in the PR is morethan a first minimum threshold, the tenth test further evaluatingwhether the total number of pixels in the ER is more than a secondminimum threshold.
 26. The method of claim 24, the at least one testcomprising a fourth test evaluating whether an ER contains more than aminimum proportion of pixels having a color falling within apredetermined color range.
 27. The method of claim 26, the predeterminedcolor range being a color range associated with a color of skin.
 28. Themethod of claim 24, the at least one test comprising a sixth testevaluating whether the pupil region of a color correction candidatecontains less than a maximum proportion of the total number of skinpixels in the eye region of the color correction candidate.
 29. Themethod of claim 24, the at least one test comprising a thirteenth testevaluating whether the number of target color pixels in a PR is greaterthan a predetermined proportion of the total number of target colorpixels in the ER.
 30. The method of claim 24, the at least one testcomprising an eighteenth test evaluating whether the number of glintpixels in an ER exceeds a predetermined maximum.
 31. The method of claim30, the predetermined maximum being a predetermined factor times thenumber of target color pixels in the ER.
 32. The method of claim 24,further comprising defining an iris region (IR), the at least one testfurther defined with respect to an iris region (IR).
 33. The method ofclaim 32, the defining the IR comprising defining the inner border ofthe IR to be adjacent the outer border of the PR, and defining the outerborder of the IR to be disposed a predetermined proportion of thedistance between the border of the PR and the border of the ER.
 34. Themethod of claim 33, the predetermined proportion of the distance beingone-half.
 35. The method of claim 32, the at least one test comprising afourteenth test evaluating whether the number of target color pixels inan IR is greater than a predetermined factor times the number of redpixels in the PR.
 36. The method of claim 32, the at least one testcomprising a sixteenth test evaluating whether the number of glint andother-color pixels in the combined PR and IR is less than apredetermined proportion of the total number of pixels in the PR and IR.37. The method of claim 32, the at least one test comprising aseventeenth test evaluating whether the number of glint pixels in the PRand IR is more than a predetermined threshold.
 38. The method of claim32, the at least one test comprising a nineteenth test evaluatingwhether the number of glint pixels in the PR and IR exceeds apredetermined threshold.
 39. The method of claim 32, further comprisingdefining a sclera region (SR), the at least one test further definedwith respect to the SR.
 40. The method of claim 39, the defining the SRcomprising defining the inner border of the SR to be adjacent to theouter border of the IR, and defining the outer border of the SR to bethe outer border of the ER.
 41. The method of claim 39, the at least onetest comprising a second test evaluating whether an inner boundary of anIR is separated from an inner boundary of an SR by a minimum distance.42. The method of claim 39, the at least one test comprising a fifthtest evaluating whether the SR of a color correction candidate containsmore than a minimum proportion of the total number of skin pixels in theER of the color correction candidate.
 43. The method of claim 39, the atleast one test comprising an eighth test evaluating whether the numberof other-color pixels in the SR of a color correction candidate is morethan a predetermined factor times the number of other-color pixels inthe IR and PR combined.
 44. The method of claim 39, the at least onetest comprising a fifteenth test evaluating whether the number of targetcolor pixels in a PR is less than a predetermined proportion of thenumber of target color pixels in the IR and SR combined.
 45. Anapparatus for adjusting eye color in a digital image, the apparatuscomprising: means for defining a plurality of color correctioncandidates, each candidate comprising a plurality of pixels; means fordefining a pupil region (PR) for each color correction candidate; meansfor performing at least one test on each of the plurality of colorcorrection candidates, the at least one test defined with respect to thepupil region (PR); means for comparing a color correction candidate withanother color correction candidate based on the result of said at leastone test; means for identifying a preferred color correction candidatebased on the comparing; and means for adjusting the color of pixels inthe preferred color correction candidate.
 46. A computer program productfor correcting eye color in a digital image, the product comprising:computer-readable medium comprising: code for causing a computer todefine a plurality of color correction candidates, each candidatecomprising a plurality of pixels; code for causing a computer to definea pupil region (PR) for each color correction candidate; code forcausing a computer to perform at least one test on each of the pluralityof color correction candidates, the at least one test defined withrespect to the pupil region (PR); code for causing a computer to comparea color correction candidate with another color correction candidatebased on the result of said at least one test; code for causing acomputer to identify a preferred color correction candidate based on thecomparing; and code for causing a computer to adjust the color of pixelsin the preferred color correction candidate.
 47. A method for adjustingeye color in a digital image, the method comprising: defining aplurality of color correction candidates, each color correctioncandidate comprising a plurality of pixels; for each color correctioncandidate, computing a match metric between the color correctioncandidate and every other color correction candidate; based on the matchmetric, assigning a complementary color correction candidate to eachcolor correction candidate, each color correction candidate and assignedcomplementary color correction candidate forming a pair; comparing acharacteristic of a first pair with a second pair to determine apreferred pair; and adjusting the color of pixels in the colorcorrection candidate and the complementary color correction candidate ofthe preferred pair.
 48. The method of claim 47, further comprisingdeassigning the assigned complementary color correction candidate if thedistance between the candidate and the assigned complementary candidateis greater than a threshold.
 49. The method of claim 47, furthercomprising deassigning the assigned complementary color correctioncandidate if the differential between the size of the candidate and thesize of the assigned complementary candidate is greater than athreshold.
 50. The method of claim 49, the computing a match metriccomprising evaluating a slope between a color correction candidate andanother color correction candidate.
 51. The method of claim 50, theassigning a complementary color correction candidate comprisingselecting a color correction candidate having the least value of slopeevaluated.
 52. The method of claim 47, the computing a match metriccomprising evaluating a distance between a color correction candidateand another color correction candidate.
 53. The method of claim 47, thecomputing a match metric comprising evaluating a number of accumulatedfailed match points of a color correction candidate.
 54. The method ofclaim 53, the computing a match metric comprising evaluating adifference between the failed match points of a candidate with thefailed match points of another candidate.
 55. The method of claim 54,the difference comprising the ratio between the failed match points of acandidate and the failed match points of another candidate.
 56. Themethod of claim 47, the computing a match metric comprising evaluatingwhether each candidate is selected as a complementary color correctioncandidate's complementary color correction candidate.
 57. The method ofclaim 47, the computing a match metric comprising: evaluating a slopebetween a color correction candidate and another color correctioncandidate; evaluating a distance between a color correction candidateand another color correction candidate; evaluating the number of failedmatch points of a color correction candidate; evaluating whether a colorcorrection candidate is selected as another color correction candidate'scomplementary color correction candidate; the match metric comprising acombination of the result of each evaluation.
 58. An apparatus foradjusting eye color in a digital image, the apparatus comprising: meansfor defining a plurality of color correction candidates, each colorcorrection candidate comprising a plurality of pixels; means for, foreach color correction candidate, computing a match metric between thecolor correction candidate and every other color correction candidate;means for, based on the match metric, assigning a complementary colorcorrection candidate to each color correction candidate, each colorcorrection candidate and associated complementary color correctioncandidate forming a pair; means for comparing a characteristic of afirst pair with a second pair to determine a preferred pair; and meansfor adjusting the color of pixels in the color correction candidate andthe complementary color correction candidate of the preferred pair. 59.A computer program product for correcting eye color in a digital image,the product comprising: computer-readable medium comprising: code forcausing a computer to define a plurality of color correction candidates,each color correction candidate comprising a plurality of pixels; codefor causing a computer to, for each color correction candidate, computea match metric between the color correction candidate and every othercolor correction candidate; code for causing a computer, to based on thematch metric, assign a complementary color correction candidate to eachcolor correction candidate, each color correction candidate andassociated complementary color correction candidate forming a pair; codefor causing a computer to compare a characteristic of a first pair witha second pair to determine a preferred pair; and code for causing acomputer to adjust the color of pixels in the color correction candidateand the complementary color correction candidate of the preferred pair.60. A method for adjusting eye color in a digital image, the methodcomprising: defining a plurality of color correction candidates, eachcandidate comprising a plurality of pixels; for each color correctioncandidate, selecting a complementary color correction candidate, thecolor correction candidate and the complementary color correctioncandidate forming a pair; assigning a failure metric to each pair;defining a first region in the digital image having a first spatialrelationship relative to a first pair; and comparing the failure metricof any pair lying in the first region to the failure metric of the firstpair.
 61. The method of claim 60, the failure metric assigned to a pairbeing a number of failed match points corresponding to the candidate inthe pair having fewer failed match points.
 62. The method of claim 60,the first region comprising a set of adjacent pixels, the first spatialrelationship defined as being spatially lower in the digital imagerelative to a rectangle enclosing the first pair.
 63. The method ofclaim 62, the rectangle enclosing the first pair being bigger than theminimum-sized rectangle enclosing the first pair.
 64. The method ofclaim 60, further comprising not adjusting the pixel color of any pairlying in the first region having a failure metric greater than thefailure metric of the first pair.
 65. The method of claim 60, furthercomprising not adjusting the pixel color of the first pair if any pairlying in the first region has a failure metric less than the failuremetric of the first pair.
 66. An apparatus for adjusting eye color in adigital image, the apparatus comprising: means for defining a pluralityof color correction candidates, each candidate comprising a plurality ofpixels; means for, for each color correction candidate, selecting acomplementary color correction candidate, the color correction candidateand the complementary color correction candidate forming a pair; meansfor assigning a failure metric to each pair; means for defining a firstregion in the digital image having a first spatial relationship relativeto a first pair; and means for comparing the failure metric of any pairlying in the first region to the failure metric of the first pair.
 67. Acomputer program product for correcting eye color in a digital image,the product comprising: computer-readable medium comprising: code forcausing a computer to define a plurality of color correction candidates,each candidate comprising a plurality of pixels; code for causing acomputer to, for each color correction candidate, select a complementarycolor correction candidate, the color correction candidate and thecomplementary color correction candidate forming a pair; code forcausing a computer to assign a failure metric to each pair; code forcausing a computer to define a first region in the digital image havinga first spatial relationship relative to a first pair; and code forcausing a computer to compare the failure metric of any pair lying inthe first region to the failure metric of the first pair.
 68. A methodfor adjusting eye color in a digital image, the method comprising:defining a plurality of color correction candidates, each candidatecomprising a plurality of target color pixels; performing a plurality oftests on each color correction candidate; for each color correctioncandidate, accumulating a number of failed match points (FMP's) based ona result of each test performed; and comparing the accumulated FMP's ofat least one color correction candidate to the accumulated FMP's of atleast one other color correction candidate.
 69. The method of claim 68,each target color pixel having a color lying within at least onepredetermined range in an HSV color space.
 70. The method of claim 68,further comprising identifying at least one preferred color correctioncandidate based on the comparing, and adjusting the color of pixels inthe at least one preferred color correction candidate.
 71. The method ofclaim 70, the identifying at least one preferred color correctioncandidate comprising identifying the color correction candidate havingthe fewest number of FMP's.
 72. The method of claim 68, furthercomprising identifying a best color correction candidate having thefewest number of FMP's based on the comparing, and not adjusting thecolor of pixels in any color correction candidate not within a thresholddistance from the best color correction candidate.
 73. The method ofclaim 68, further comprising identifying a best color correctioncandidate having the fewest number of FMP's based on the comparing, andnot adjusting the color of pixels in any color correction candidate notwithin a threshold vertical distance from the vertical position in theimage of the best color correction candidate.
 74. The method of claim68, further comprising, if the accumulated FMP's of one color correctioncandidate are equal to the accumulated FMP's of another color correctioncandidate, selecting the color correction candidate higher in the imagefor further processing.
 75. The method of claim 74, the furtherprocessing comprising adjusting the color of pixels in the selectedcolor correction candidate.
 76. The method of claim 75, furthercomprising classifying each pixel in each color correction candidateinto a pixel type, the pixel type being target color, glint, skin, orother-color.
 77. The method of claim 75, the plurality of tests definedwith respect to a plurality of regions within each color correctioncandidate.
 78. The method of claim 77, the plurality of regions withineach color correction candidate comprising an eye region (ER), a scleraregion (SR), an iris region (IR), and a pupil region (PR).
 79. Themethod of claim 78, the eye region comprising the entire regionassociated with a color correction candidate, the pupil regioncomprising an innermost region within the eye region, the iris regioncomprising a region surrounding the pupil region, and the sclera regioncomprising a region surrounding the iris region.
 80. The method of claim68, the accumulating a number of FMP's based on a result of each testperformed comprising accumulating a number of FMP's if the result of atest is true.
 81. A method for adjusting eye color in a digital image,the method comprising: step for defining a plurality of color correctioncandidates, each candidate comprising a plurality of pixels; step forassigning a number of failed match points to each of the plurality ofcolor correction candidates; comparing the accumulated FMP's of at leastone color correction candidate to the accumulated FMP's of at least oneother color correction candidate.
 82. An apparatus for adjusting eyecolor in a digital image, the apparatus comprising: means for defining aplurality of color correction candidates, each candidate comprising aplurality of target color pixels; means for performing a plurality oftests on each color correction candidate; means for, for each colorcorrection candidate, accumulating a number of failed match points(FMP's) based on a result of each test performed; and means forcomparing the accumulated FMP's of at least one color correctioncandidate to the accumulated FMP's of at least one other colorcorrection candidate.
 83. A computer program product for correcting eyecolor in a digital image, the product comprising: computer-readablemedium comprising: code for causing a computer to define a plurality ofcolor correction candidates, each candidate comprising a plurality oftarget color pixels; code for causing a computer to perform a pluralityof tests on each color correction candidate; code for causing a computerto, for each color correction candidate, accumulate a number of failedmatch points (FMP's) based on a result of each test performed; and codefor causing a computer to compare the accumulated FMP's of at least onecolor correction candidate to the accumulated FMP's of at least oneother color correction candidate.
 84. A method for adjusting eye colorin a digital image, the method comprising: defining a plurality of colorcorrection candidates, each candidate comprising a plurality of pixels;computing an average size of the color correction candidates;identifying a color correction candidate having a size outside a rangethat is a function of the average size; and not adjusting the color ofpixels in said identified color correction candidate.
 85. The method ofclaim 84, the range having a minimum that is a proportion of the averagesize, and a maximum that is a predetermined number times the averagesize.
 86. A method for adjusting eye color in a digital image, themethod comprising: identifying target color pixels in the digital image;defining color correction candidates based on the target color pixels;defining a region map for each color correction candidate, the regionmap comprising at least one of an eye region, a pupil region, an irisregion, and a sclera region; computing statistics for each colorcorrection candidate based on its region map; identifying preferredcolor correction candidates based on the computed statistics; andadjusting the color of pixels in the preferred color correctioncandidates.
 87. The method of claim 86, the target color pixels beingpixels having a color in a range associated with red.
 88. The method ofclaim 86, the adjusting the color of pixels comprising adjusting thesaturation of a target color pixel as:Saturation_(new) =P _(red)*Saturation_(old)+(1−P_(red))*Saturation_(old) /n.
 89. The method of claim 88, wherein n is 2.90. The method of claim 86, the defining color correction candidatescomprising: identifying sets of connected pixels in the target colorpixels; defining a set of regular structures, each regular structureassociated with a set of connected pixels; computing likelihoodcharacteristics for each of the regular structures; and identifying,based on the likelihood characteristics, a likely subset of the set ofregular structures; and defining color correction candidates associatedwith regular structures in the likely subset.
 91. The method of claim90, the identifying, based on the likelihood characteristics, a likelysubset of the set of regular structures comprising eliminating regularstructures identified as unlikely.
 92. The method of claim 90, thecomputing likelihood characteristics comprising identifying as unlikelythose regular structures having dimensions predetermined to be unlikely.93. The method of claim 90, further comprising eliminating a set ofregular structures having a mutual distance less than a predetermineddistance threshold, and defining a new regular structure associated withthe set of eliminated regular structures.
 94. The method of claim 90,the computing likelihood characteristics comprising identifying asunlikely the smaller or smallest regular shape within sets of regularshapes that share at least one overlapping pixel.
 95. The method ofclaim 90, the regular structure being a minimum-size rectangle thatentirely encloses the associated set of connected pixels.
 96. The methodof claim 90, the defining color correction candidates comprising:expanding each regular structure within the likely subset by anexpansion factor; searching a search area larger than each expandedregular structure for skin pixels; eliminating expanded regularstructures whose search areas do not contain a minimum threshold of skinpixels; and defining color correction candidates corresponding toexpanded regular structures remaining after the step of eliminating. 97.The method of claim 86, the statistics for each color correctioncandidate based on the distribution of colors within each candidate'sregion map.
 98. The method of claim 86, further comprising, for eachcolor correction candidate, identifying a candidate pair comprising thatcolor correction candidate and another color correction candidate, andeliminating color correction candidates based on characteristics of theassociated candidate pairs.
 99. The method of claim 98, the eliminatingcolor correction candidates based on characteristics of the associatedcandidate pairs comprising: eliminating color correction candidates ofan associated candidate pair if their separation from each other isabove a threshold.
 100. The method of claim 98, the eliminating colorcorrection candidates based on characteristics of the associatedcandidate pairs comprising: identifying a first pair lying in a mouthregion of a second pair; eliminating color correction candidates of afirst pair if the first candidate pair is otherwise less likely than thesecond candidate pair; and eliminating color correction candidates ofthe second candidate pair if the first candidate pair is otherwise morelikely than the second candidate pair.
 101. The method of claim 86,further comprising eliminating a color correction candidate having asize larger than a first function of the average color correctioncandidate size, and eliminating a color correction candidate having asize smaller than a second function of the average color correctioncandidate size.
 102. An apparatus for adjusting eye color in a digitalimage, the apparatus comprising: means for identifying target colorpixels in the digital image; means for defining color correctioncandidates based on the target color pixels and eliminating colorcorrection candidates based on a set of criteria; means for defining aregion map for each color correction candidate, the region mapcomprising at least one of an eye region, a pupil region, an irisregion, and a sclera region; and means for computing statistics for eachcolor correction candidate based on each region map; means foridentifying at least one preferred color correction candidate based onthe computed statistics; and means for adjusting the color of pixels inthe at least one preferred color correction candidate.
 103. A computerprogram product for correcting eye color in a digital image, the productcomprising: computer-readable medium comprising: code for causing acomputer to identify target color pixels in the digital image; code forcausing a computer to define color correction candidates based on thetarget color pixels; code for causing a computer to define a region mapfor each color correction candidate, the region map comprising at leastone of an eye region, a pupil region, an iris region, and a scleraregion; code for causing a computer to compute statistics for each colorcorrection candidate based on each region map; code for causing acomputer to identify final color correction candidates based on thecomputed statistics; code for adjusting the color of pixels in the finalcolor correction candidates.